1. Field of the Invention
The present invention relates to an image processing device, a plane detection method, and a recording medium upon which a plane detection program is recorded, which, using a distance image, perform detection of planes which correspond to walls, a floor or a road surface, as required when a robot or vehicle moves.
2. Description of the Related Art
In the prior art, it is general to use Hough conversion as a method of detecting straight lines in a two dimensional plane. Accordingly, in order to detect planes in a three dimensional space by applying this method, it is necessary to sort (ballot) all the points in the three dimensions in a parameter space, and the problem arises that the amount of memory used for this sorting becomes inflated. Furthermore, since sorting is also performed for parameters of surfaces which do not exist in practice, the further problem arises that a very large number of candidates for planes exist, so that a great amount of processing time is required for the plane detection processing.
The present invention has been conceived in view of this sort of circumstance, and its objective is to propose an image processing device, a plane detection method, and a recording medium upon which a plane detection program is recorded, which can perform the required detection of planes in a simple manner when a robot or vehicle moves autonomously.
The invention of the first aspect comprises: a distance image capture device (for example, the distance image capture section 1 of the embodiment) which captures a distance image based upon three dimensional coordinate values obtained for various points upon body surfaces by measuring the distances to said points upon said body surfaces in the forward visual field of a distance sensor; a straight line extraction device (for example, the steps S1 through S6 performed by the vertical plane extraction section of the embodiment) which obtains three dimensional coordinate values of two points from a viewpoint picture element in said distance image and a picture element which is separated by a predetermined distance from said viewpoint picture element, and which extracts a plurality of straight lines in a horizontal plane using two dimensional coordinate values by ignoring the vertical component among these three dimensional coordinate values; and a plane detection device (for example, the step 7 performed by the vertical plane extraction section of the embodiment) which obtains points upon said body surfaces which lie upon various straight lines which have been extracted by said straight line extraction device, and which detects points which lie upon the same straight line as being upon the same vertical plane.
According to the invention of the first aspect since it is arranged that when detecting the vertical planes the calculation is performed using only the X and the Y components but not using the Z components, and that the vertical planes are extracted by specifying straight lines in the X-Y plane, therefore the beneficial effects are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Further, since the number of sorts into the Hough space is reduced by using the mask, thereby the beneficial effects are obtained that it is possible to shorten the processing time, and moreover that it is possible to extract the planes with good accuracy.
The invention of the second aspect comprises a distance image capture device (for example, the distance image capture section 1 of the embodiment) which captures a distance image based upon three dimensional coordinate values obtained for various points upon body surfaces by measuring the distances to said points upon said body surfaces in the forward visual field of a distance sensor; a plane specification device (for example, the steps S11 through S13 and S18 which are performed by the horizontal plane extraction section 5 of the embodiment) which obtains three dimensional coordinate values of three points from a viewpoint picture element in said distance image and two picture elements which are separated by predetermined distances from said viewpoint picture element, and which specifies a plane existing in three dimensional space from these three dimensional coordinate values; and a plane detection device (for example, the steps S14 and S19 which are performed by the horizontal plane extraction section 5 of the embodiment) which detects a plane specified by said plane specification device of which the inclination of the perpendicular vector is within a predetermined angle as a road surface or a floor surface.
According to the invention of second aspect, since when detecting horizontal planes which correspond to a floor surface or to a road surface, among the candidates for the planes which are to be detected, plane detection is performed only using those candidates for which the angle of the perpendicular vector is within a predetermined range, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy. Yet further, it is also possible to detect slopes or the like which have small inclination as being floor surfaces or road surfaces.
The invention of the third aspect is a vehicle movement control device comprising an image processing device according to the first or second aspect of the invention, wherein said vehicle movement control device comprises a movement control device (for example, the movement control section 8 of the embodiment) which, during vehicle movement, performs movement control by referring to the results of detection by said plane detection device.
According to the invention of the third aspect, since it is arranged that the movement of the vehicle is controlled while referring to the plane detection image and to the distance image, thereby the benefits are obtained that it is possible to control the position of the vehicle with great accuracy, and also that, because it becomes possible to shorten the processing time, the burden of path selection processing upon the movement control device can be reduced.
The invention of the fourth aspect is an autonomously moving robot comprising an image processing device according to the first or second aspect, wherein said autonomously moving robot comprises a movement control device (for example, the movement control section 8 of the embodiment) which, during robot movement, performs movement control by referring to the results of detection by said plane detection device.
According to the invention of the fourth aspect, since it is arranged that the movement of the robot is controlled while referring to the plane detection image and to the distance image, thereby the benefits are obtained that it is possible to control the position of the robot with great accuracy, and also that, because it becomes possible to shorten the processing time, the burden of path selection processing upon the robot can be reduced.
The invention of the fifth aspect comprises distance image capture processing which captures a distance image based upon three dimensional coordinate values obtained for various points upon body surfaces by measuring the distances to said points upon said body surfaces in the forward visual field of a distance sensor; straight line extraction processing (for example, the steps S1 through S6 performed by the vertical plane extraction section 3 of the embodiment) which obtains three dimensional coordinate values of two points from a viewpoint picture element in said distance image and a picture element which is separated by a predetermined distance from said viewpoint picture element, and which extracts a plurality of straight lines in a horizontal plane using two dimensional coordinate values by ignoring the vertical component among these three dimensional coordinate values; and plane detection processing (for example, the step S7 performed by the vertical plane extraction section 3 of the embodiment) which obtains points upon said body surfaces which lie upon various straight lines which have been extracted by said straight line extraction processing, and which detects points which lie upon the same straight line as being upon the same vertical plane.
According to the invention of the fifth aspect, since it is arranged that, when detecting vertical planes which correspond to walls, the calculations are performed only using the X and Y components but not using the Z component, and the vertical planes are extracted by specifying straight lines in the X-Y plane, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy.
The invention of the sixth aspect comprises distance image capture processing which captures a distance image based upon three dimensional coordinate values obtained for various points upon body surfaces by measuring the distances to said points upon said body surfaces in the forward visual field of a distance sensor; plane specification processing (for example, the steps S11 through S13 and S18 performed by the horizontal plane extraction section 5 of the embodiment) which obtains three dimensional coordinate values of three points from a viewpoint picture element in said distance image and two picture elements which are separated by predetermined distances from said viewpoint picture element, and which specifies a plane existing in three dimensional space from these three dimensional coordinate values; and plane detection processing (for example, the steps S14 and S19 performed by the horizontal plane extraction section 5 of the embodiment) which detects a plane specified by said plane specification processing of which the inclination of the perpendicular vector is within a predetermined angle as a road surface or a floor surface.
According to the invention of the sixth aspect, since when detecting horizontal planes which correspond to a floor surface or to a road surface, among the candidates for the planes which are to be detected, plane detection is performed only using those candidates for which the angle of the perpendicular vector is within a predetermined range, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy. Yet further, it is also possible to detect slopes or the like which have small inclination as being floor surfaces or road surfaces.
The invention of the seventh aspect is a recording medium which can be read by a computer, upon which is recorded a plane detection program which detects planes in a forward visual field, wherein said plane detection program causes said computer to perform: distance image capture processing which captures a distance image based upon three dimensional coordinate values obtained for various points upon body surfaces by measuring the distances to said points upon said body surfaces in the forward visual field of a distance sensor; straight line extraction processing (for example, the steps S1 through S6 performed by the vertical plane extraction section 3 of the embodiment) which obtains three dimensional coordinate values of two points from a viewpoint picture element in said distance image and a picture element which is separated by a predetermined distance from said viewpoint picture element, and which extracts a plurality of straight lines in a horizontal plane using two dimensional coordinate values by ignoring the vertical component among these three dimensional coordinate values; and plane detection processing (for example, the step S7 performed by the vertical plane extraction section 3 of the embodiment) which obtains points upon said body surfaces which lie upon various straight lines which have been extracted by said straight line extraction processing, and which detects points which lie upon the same straight line as being upon the same vertical plane.
According to the invention of the seventh aspect, since it is arranged that, when detecting vertical planes which correspond to walls, the calculations are performed only using the X and Y components but not using the Z component, and the vertical planes are extracted by specifying straight lines in the X-Y plane, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy.
The invention of the eighth aspect is a recording medium which can be read by a computer, upon which is recorded a plane detection program which detects planes in a forward visual field, wherein said plane detection program causes said computer to perform: distance image capture processing which captures a distance image based upon three dimensional coordinate values obtained for various points upon body surfaces by measuring the distances to said points upon said body surfaces in the forward visual field of a distance sensor; plane specification processing (for example, the steps S11 through S13 and S18 performed by the horizontal plane extraction section 5 of the embodiment) which obtains three dimensional coordinate values of three points from a viewpoint picture element in said distance image and two picture elements which are separated by predetermined distances from said viewpoint picture element, and which specifies a plane existing in three dimensional space from these three dimensional coordinate values; and plane detection processing (for example, the steps S14 and S19 performed by the horizontal plane extraction section 5 of the embodiment) which detects a plane specified by said plane specification processing of which the inclination of the perpendicular vector is within a predetermined angle as a road surface or a floor surface.
According to the invention of the eighth aspect, since when detecting horizontal planes which correspond to a floor surface or to a road surface, among the candidates for the planes which are to be detected, plane detection is performed only using those candidates for which the angle of the perpendicular vector is within a predetermined range, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy. Yet further, it is also possible to detect slopes or the like which have small inclination as being floor surfaces or road surfaces.
As explained above, since according to the inventions described in the first, fifth, and seventh aspects it is arranged that, when detecting vertical planes which correspond to walls, calculations are performed only using the X and Y components but not using the Z component, and the vertical planes are extracted by specifying straight lines in the X-Y plane, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy.
Yet further, since according to the inventions described in the second, sixth, and eighth aspects it is arranged that, when detecting horizontal planes which correspond to a floor surface or to a road surface, among the candidates for the planes which are to be detected, plane detection is performed only using those candidates for which the angle of the perpendicular vector is within a predetermined range, thereby the benefits are obtained that it is possible to reduce the amount of memory which is used, and moreover that it is possible to shorten the processing time. Furthermore, since the number of sorts into the Hough space is reduced using the mask, the benefits are obtained that it is possible further to reduce the processing time, and moreover that it is possible to extract the planes with good accuracy. Yet further, it is also possible to detect slopes or the like which have small inclination as being floor surfaces or road surfaces.
Even further, since according to the invention of the third aspect it is arranged that the movement of the vehicle is controlled while referring to the plane detection image and to the distance image, thereby the benefits are obtained that it is possible to control the position of the vehicle with great accuracy, and also that, because it becomes possible to shorten the processing time, the burden of path selection processing upon the movement control device can be reduced.
Even yet further, since according to the invention of the fourth aspect it is arranged that the movement of the robot is controlled while referring to the plane detection image and to the distance image, thereby the benefits are obtained that it is possible to control the position of the robot with great accuracy, and also that, because it becomes possible to shorten the processing time, the burden of path selection processing upon the robot can be reduced.